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WHAT IS CLAIMED IS: 

1. An annotating system for aiding a user in 
performing bulk mapping of queries to tasks to obtain 
training data for training a search component, the 
annotating system comprising: 

a query log containing queries which have 
previously been submitted to a search 
engine ; 

a task list containing a plurality of possible 
tasks ; 

a machine learning component which suggests best 
guess query- to- task mappings as a function 
of the training data; and 

a graphical user interface generating component 
configured to display at least some of the 
plurality of query entries in the query log 
and at least some of the plurality of tasks 
in the task list in a manner which 
associates the suggested best guess query- 
to-task mappings. 

2. The annotating system of claim 1, wherein the 
machine learning component is configured to suggest, 
for each of a plurality of query entries 
corresponding to the query log, a best guess task for 
potential query-to-task mapping as a function of the 
training data. 
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3. The machine learning component of claim 2, 
wherein the graphical user interface generating 
component is configured to display the plurality of 
query entries in the query log in a manner which 
associates each of the displayed plurality of query 
entries with its corresponding suggested best guess 
task. 

4. The annotating system of claim 3, wherein the 
graphical user interface generating component is 
further configured to receive a first type of input 
from the user when the user wishes to map a 
particular query entry to its corresponding suggested 
best guess task, and wherein upon receipt of the 
first type of input through the graphical user 
interface the machine learning component updates the 
training data by mapping the particular query entry 
to the suggested best guess task. 

5. The annotating system of claim 4, wherein the 
graphical user interface generating component is 
further configured to receive a second type of input 
from the user when the user wishes to view a list of 
next best guess tasks for the particular query entry 
for potential mapping. 
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6. The annotating system of claim 4, wherein each 
of the plurality of query entries corresponding to 
the query log is a query bundle, and wherein each 
query bundle represents a plurality of queries in the 
query log which have been bundled together. 

7. The annotating system of claim 6, wherein upon 
receipt of the first type of input through the 
graphical user interface the machine learning 
component updates the training data by mapping each 
of the plurality of queries, represented by the query 
bundle, to the suggested best guess task. 

8. The annotating system of claim 6, wherein the 
suggested best guess for each query bundle is a 
weighted average of best guesses for each of . the 
plurality of queries represented by the query bundle. 

9. The annotating system of claim 6, wherein the 
graphical user interface generating component is 
further configured to receive a second type of input 
from the user when the user wishes to view a list of 
queries represented by a particular query bundle, and 
in response to receipt of the second type of input 
the graphical user interface generating component 
being configured to display the list of queries 
represented by the query bundle. 
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10. The annotating system of claim 4, wherein the 
graphical user interface generating component is 
further configured to display the task list. 

11. The annotating system of claim 10, wherein the 
graphical user interface generating component is 
further configured to receive a second type of input 
from the user when the user wishes to map the 
particular query entry to a task, from the displayed 
task list, which differs from the suggested best 
guess task, and wherein upon receipt of the second 
type of input through the graphical user interface 
the machine learning component is configured to 
update the training data by mapping the particular 
query entry to the task from the displayed task list. 

12. The annotating system of claim 4, wherein upon 
updating the training data by mapping the particular 
query entry to a task, the machine learning component 
is configured to automatically update the best guess 
task for each of the remaining ones of the plurality 
of query entries as a function of the updated 
training data. 

13. The annotating system of claim 12, wherein the 
machine learning component is a Naive Bayes 
classifier. 
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14. The annotating system of claim 1, wherein the 
machine learning component is configured to suggest, 
for each of the plurality of possible tasks in the 
task list, a list of queries from the query log which 
are probable for query-to-task mapping as a function 
of the training data. 

15. A method of aiding a user in performing bulk 
mapping of queries to tasks to obtain training data 
for training a search component, the method 
comprising: 

obtaining a query log containing queries which 
have previously been submitted to a search 
engine ; 

obtaining a task list containing a plurality of 
possible tasks; 

determining for each of a plurality of query 
entries corresponding to the query log a 
best guess task for potential query-to-task 
mapping, wherein the best guess task is 
determined as a function of the training 
data using a machine learning component; 
and 

displaying the plurality of query entries in the 
query log in a manner which associates each 
of the displayed plurality of query entries 
with its corresponding suggested best guess 
task. 
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16. The method of claim 15, and further comprising: 
receiving a first type of input from the user 

when the user wishes to map a particular 
query entry to its corresponding suggested 
best guess task; and 
updating the training data using the machine 
learning component, by mapping the 
particular query entry to the suggested 
best guess task, upon receipt of the first 
type of input. 

17. The method of claim 16, and further comprising: 
receiving a second type of input from the user 

when the user wishes to view a list of next 
best guess tasks for the particular query 
entry for potential mapping; and 
displaying the list of next best guess tasks, 
determined as functions of the training 
data using the machine learning component, 
for the particular query entry. 

18. The method of claim 16, wherein each of the 
plurality of query entries corresponding to the query 
log is a query bundle, and wherein each query bundle 
represents a plurality of queries in the query log 
which have been bundled together. 
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19. The method of claim 18, wherein upon receipt of 
the first type of input the method further 
comprising: 

updating the training data, using the machine 
learning component, by mapping each of the 
plurality of queries represented by the 
query bundle to the suggested best guess 
task. 

20. The method of claim 18, and further comprising: 
receiving a second type of input from the user 

when the user wishes to view a list of 
queries represented by a particular query 
bundle; and 

displaying the list of queries represented by 
the query bundle in response to receipt of 
the second type of input. 

21. The method of claim 16, and further comprising 
displaying the task list. 
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22. The method of claim 21, and further comprising: 
receiving a second type of input from the user 

when the user wishes to map the particular 
query entry to a task, from the displayed 
task list, which differs from the suggested 
best guess task; and 
updating the training data, using the machine 
learning component, by mapping the 
particular query entry to the task from the 
displayed task list upon receipt of the 
second type of input . 

23. The method of claim 16, and upon updating the 
training data by mapping the particular query entry 
to a task, further comprising automatically updating 
the best guess task, using the machine learning 
component, for each of the remaining ones of the 
plurality of query entries as a function of the 
updated training data. 

24. The method of claim 23, wherein the machine 
learning component is a Naive Bayes classifier. 

25. The method of claim 15, wherein determining for 
each of the plurality of query entries corresponding 
to the query log a best guess task further comprises 
determining, for each task in the task list, a list 
of queries from the query log which are probable for 
query-to-task mapping as a function of the training 
data using the machine learning component. 
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26. A computer readable medium containing computer 
executable instructions for implementing the steps of 
claim 15. 

27. A method of performing bulk mapping of queries 
to tasks to obtain training data for training a 
search component, the method comprising: 

downloading a copy of a classifier model from a 

server to a local computer; 
performing query- to- task mapping at the local 

computer based upon guesses generated using 

the downloaded classifier model; 
updating a local training data store based upon 

the performed query- to- task mapping at the 

local computer; and 
updating a server training data store, used to 

create the classifier model, with the local 

training data store. 

28. The method of claim 27, and further comprising 
updating the classifier model on the server using the 
updated server training data store. 

29. The method of claim 27, and further comprising 
repeating the steps of the method for each of a 
plurality of local computers such that the server 
training data store is updated using local training 
data stores from each of the plurality of computers. 
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30. A computer readable medium containing computer 
executable instructions for implementing the steps of 
claim 27. 



